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Abstract. We define Hilbert-Siegel modular forms and Hecke "operators" acting 
on them. As with Hilbert modular forms (i.e. with Siegel degree 1), these linear 
transformations are not linear operators until we consider a direct product of spaces 
of modular forms (with varying groups) , modulo natural identifications we can make 
between certain spaces. With Hilbert-Siegel forms (i.e. with arbitrary Siegel degree) 
we identify several families of natural identifications between certain spaces of mod- 
ular forms. We associate the Fourier coefficients of a form in our product space to 
even integral lattices, independent of basis and choice of coefficient rings. We then 
determine the action of the Hecke operators on these Fourier coefficients, paralleling 
the result of Hafner and Walling for Siegel modular forms (where the number field is 
the field of rationals). 



1. Introduction. 

A Siegel modular form F of degree n over the rationals has a Fourier series 
supported on even integral symmetric n x n matrices. An even integral symmetric 
matrix can be interpreted as the matrix for a quadratic form on an even integral 
lattice, relative to some Z-basis for that lattice. Given the transformation property 
of F under the symplectic group, the coefficient of F attached to a matrix T is 
equal to that attached to the conjugate t GTG where G is any integral change of 
basis matrix (with determinant 1 when k, the weight of the modular form, is odd). 
Consequently we can rewrite F as a "Fourier series" supported on even integral 
lattices, without specifying a basis for each lattice. For each prime p there are 
n+ 1 Hecke operators, T(p) and Tj(p 2 ) (1 < j < n) associated to p, n of which are 
algebraically independent. In [5] we determined the action of these operators on 
the Fourier coefficients of F. In this paper we extend this result to Hilbert-Siegel 
modular forms. 

With K a totally real number field and V a prime ideal, we mimic the construc- 
tion of the classical Hecke operators and construct a linear transformation T(V) 
acting on Hilbert modular forms. When V is not principally generated, T(V) maps 
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modular forms attached to V = SL 2 (0) (O the ring of integers of K), to forms 
attached to the "psuedo-conjugate" 

(o ?) r (V !) = {(' 2)= «.'e ,»e P ,ce P - = 

So for T('P) to be a linear operator (meaning its domain and codomain are equal) , it 
is necessary to consider a (finite) direct product of spaces of modular forms attached 
to psuedo-conjuagates of SL 2 (0). In [7], Shimura defined "Fourier coefficients" 
attached to integral ideals of a form in this direct product, and he determined the 
action of T(V) on these Fourier coefficients. 

In the case of Hilbert-Siegel modular forms, we need to consider a (finite) direct 
product of spaces of forms attached to psuedo-conjugates of Sp n (0) for the maps 
T(V) and Tj(V 2 ) to be linear operators. For a form in this direct product, we define 
"Fourier coefficients" attached to even integral lattices, independent of basis and 
choices of coefficient rings (note that an O-lattice is not necessarily free, and there 
are numerous ways to write it as A\X\ © • • • © A n x n with the Ai fractional ideals). 
Then we determine the action of the Hecke operators on these Fourier coefficients. 
When k is odd, we need to impose an orientation on A. Thus 

F(T) = $>(A)e*{Ar} 

clsA 

where clsA runs over isometry classes of lattices A, and e*{Ar} = 
X^G ex P (niTr( t GTGr)); here A = AiXi © ••• © A n x n , T = (B(xi,Xj)) where 
B is the symmetric bilinear form associated to the quadratic form Q on A so 
that Q(x) = B(x,x), and G varies over 0(A)\GL n (Z) when k is even, and over 
+ (A)\SL n (Z) when k is odd. (Two lattices A, Q are in the same isometry class if 
there is an isomorphism from one onto the other that preserves the quadratic form. 
Also, 0(A) is the orthogonal group of A.) 

We begin by defining symplectic groups F(Ii, . . . , X n ;,7) for fractional ideals 
1i, J ■ We show that the spaces of modular forms associated to T(Xi, . . . , X n ; J~) and 
r(X{, . . . , X^; J') are naturally isomorphic whenever cls(Xi • • -X n ) = cls(X{ • • -X' n ) 
and clx + J = clx + J' . (Here clsX denotes the wide ideal class of X, and clx+,7 
denotes the strict ideal complex of J. Thus clsX = clsX' if X = aX' for some a e K, 
and clx+,7 = clx+,7' if J = aI 2 J' for some fractional ideal X and a ^> 0.) We set 
A4k = rixj j--Mfc(r(Xi, . . . ,T n ;J))/ ~ (so we identify spaces that are naturally 
isomorphic). Next we attach the Fourier coefficients of (the components of) F 
to even integral lattices, independent of the basis and the coefficient rings used 
to realize each lattice. (For a full discussion of this, see the discussion preceeding 
Proposition 2.2.) In §3 we introduce operators S(Q) attached to fractional ideals <2, 
and we decompose M.^ as @ x M.k(x) where x varies over ideal class characters, and 
F\S(Q) = x(Q)F f° r F e -Mfc(x)- Then in §4 we introduce the Hecke operators 
T(V) and Tj(V 2 ), < j < n, and we find coset representatives giving the action of 
the operators. When then analyzing the action of the Hecke operators Tj(V 2 ) in 
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§5 we encounter incomplete character sums; we complete these by replacing Tj(V 2 ) 
with Tj(P 2 ), a combination of T e (V 2 ), 0<£<j. Finally, we show that for A J an 
even integral lattice and F e Mk(x)i the A^th coefficient of F\Tj(V 2 ) is 

AT(P)^^' A ) x (P) e ^"' A )a J (0,A)c F (0^) 

•pACfiCp-iA 

where Ej(Q, A) and ej(Q, A) are given by formulas in terms of the invariant factors 
{O : A}, and aj(Q, A) reflects some geometry of (O n A) /V(Q + A). (A formula for 
ccj(Q,A) is given at the end of §5.) A similar but much simpler argument shows 
that the A^th coefficient of F\T(V) is 

N(V) E ^c F (Q JV ") 

•PACHCA 

(see Theorem 5.2). 

In §6 we present a lemma on completing a symmetric coprime pair to a symplectic 
matrix. The reader is referred to [6] for basic results on lattices and quadratic forms. 
The authors are thankful for the referee's careful reading. 

2. Definitions, isomorphisms, and Fourier 
coefficients attached to even integral lattices. 

Let K be a totally real number field of degree d over Q, and let d denote the 
different of K. Let H( n ) denote degree n Siegel upper half-space; so 

U {n) = {r = X + iY : X, Y G M n ' n are symmetric, Y > } . 

For fractional ideals Zi, . . . ,X n , J, let 

V{X U ... ,l n ;J) = { ^ ^GG 2n (I): A t B,C t D symmetric, 

A t D-B*C = uI, u E £>+, aij E F t l~\ 

(Here A = (a*,- ), etc.) So with T = T(0, ...,£>;£>), T(l u . . . ,T n ; J) corresponds 
to the formal conjugate 5T(5 _1 where 

/JXi \ 

<■ yJF n 

0. = n — 1 



V 
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Also notice that r(Zi, . . . ,X n ; JX 2 ) = T{X 1 X, . . . , J n J; J). 

Definition. A degree n (n > 1), weight k Hilbert-Siegel modular form for 
r(Ji, . . . ,X n ;J) is a function / : H-f^ — > C so that the following two conditions 
hold. 

(1) / is "analytic on 7~tf n j and at infinity," meaning that for r G 7~tf n y 

f(r) = J2c(T)e{Tr} 

T 

where T runs over symmetric, positive semi-definite n x n matrices. Also, 
a(M) denotes the trace of a matrix M, Tr denotes the trace from K to <2, 
and 

e{Tr} = exp(7rza(Tr(Tr))). 
Here Tr(Tr) = E 1= i tWt ^ where tW is the 

image of T under the ith em- 
bedding of IK into R. 

(2) For all M E T(X U ... ,X n ;J), f\M = f where, for any matrix 
(written in n x n blocks), we define 




= det(iV(A l D - B *C)) fc / 2 det(iV(CV + D))~ k 
f({Ar + B)(Cr + D)- 1 ). 

Here denotes the norm from K to Q, extended so that 

d 

N(Ct + D) = Y[C^Ti + D^. 

i=l 

Let A- / f/ c (r(Xi, . . . ,X n ; J)) denote the space of Hilbert-Siegel modular forms for 
r(Xi, . . . ,X n ; J), and let / be a modular form in this space. Since f(r + B) = /(r) 
for all symmetric B e (X^Xj Jd~ x ), d the different of K, we must have e{TB} = 1 
for all T G supp/. Note that for T = (%), B = (bij) symmetric matrices, 

n 

a(TB) = y^tiibg + ^2 2t ij b ij- 

i=l l<i<j<n 

Thus for T G supp/, we must have T G ((/j/jj") -1 ) with T even, meaning tu G 

2/- 2 .7- 1 . 

Definitions. We define families of isomorphisms between spaces of modular forms 
as follows: Fix / G M k {T(X u . . . ,X n ;J)). 
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First, for a£l x and 1 < i < n, let 



M = I a' 1 

\ In-l 

and define 

f\U i {a) = f\[ M tM _ x 

Since( M tM _ 1 ^T{T l ,...,T w J)( M 1 t M ) = r(Ji, . . . , X n ; J) where 



X' 



Xj if £ ^ i 
aXg if £ = i, 



Ui(a) defines an isomorphism from .Mfc(r(Xi, . . . , X n ; J)) onto 
M k (T(I{,...,I' n ;J)). 

Fora>0, define W (a) : M k (T(l u . . . ,l n ;J)) -> A4 fc (r(Zi, . . . ,X n ;aJ)) by 



/|W r (a) = /| 



n 



One easily checks (as we did above for Ue(ct)) that W(a) is an isomorphism. 
For Q a fractional ideal, 1 < £ < n, choose 



A e 



Q- 1 QEiTT+i 
e+i 



so that det A = 1 (possible by Strong Approximation; see p. 42 [6]). Let 

(h-i 
M= A 

\ In-i-l 

and define 

f\V t {Q) = f\( M tM _ x 

Since ( M tM _ 1 ^T{Z> 1 ,...a'n,J)( M ~ 1 t M ) = r(Ji, • • • ,X n ; .7) where 

X, if i ^ £, £ + 1 

x; = <( aie ifi = e 

Q-^+i ifz = £ + l, 
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the map Vi(a) defines an isomorphism from M. k (T(Xi, . . . ,X n ; J)) onto 
M k (T(X{,...,X' n ;J)). 

For Q a fractional ideal and 1 < £ < j < n, set 

V ej (Q) = V £ (Q)V £+1 (Q) ■ ■ ■ Vj.^Q). 

Then Vgj(Q) defines an isomorphism from M k (Xi 7 ■ ■ ■ ,X n ; J)) onto 
A* fc (r(Jj,...,j; ;t 7)),where 

f X, Mi^l,3 

x[ = \ Qi t ifi = e . 

Proposition 2.1. X7ie maps Ut{a), W(a), Vij(Q) commute and for fixed £,j, 
these operators are multiplicative (as functions on fractional ideals). 

Proof. The tedious aspect of proving such relations among our isomorphisms is that, 
for any of the above listed maps, the domain and codomain differ. Keeping track 
of appropriate domains and codomains, and using the matrices we used to define 
the actions of these operators, it is then straightforward to verify the operators 
commute, remembering that if MTV -1 G T for any group V = F(Xi, . . . ,X n ;J'), 
then f\M = f\N for / G M k (T). □ 

Definition. For / G A* fc (r(Zi,... ,X n ;J)), g G M k (X[,... ,X' n ,J')), define the 
equivalence relation ~ by / ~ g if some composition of the maps Ui, W, Vij takes / 
to g. We define 

M k = Y[M k (T(I 1 ,...,I n ;J))/~ 

where X\,... ,X n ,J~ vary over all fractional ideals. Note that ~ partitions the 
spaces A^fc(r(Xi, . . . ,T n ; J)) according to cls(Xi • • -X n ), clx+J". Thus M k 
ricisx ci x + j- •A / tfc(r(C, ... , O, X; clsX runs over all ideal classes and cls + J runs 
over all strict ideal class complexes. {J ', J' are in the same strict ideal class complex 
if J' — aX 2 J for some fractional ideal X and a 3> 0.) 

Given an element F G M k , we can associate the Fourier coefficients of F with 
lattices equipped with positive semi-definite, even integral quadratic forms as de- 
scribed below. 

First note the following. Say / G A4 k (T(Xi, . . . ,X n ;J')) is a component of a 
chosen representative for F. So the support of / lies in (^(X i XjJ')~ 1 ^ . For T G 
supp/, consider T as defining a quadratic form Q on Ox\ © • • • © Ox n (recall that 
T is symmetric and even, meaning the ith diagonal entry of T lies in 2X~ 2 J~ 1 ). 
Thus with A = X x x x © • • • ©X n x n , Q(A) C 2J' 1 . If aO = J then the scaled lattice 
A a is even integral, meaning aQ(A) C 20. When J is not principal, we abuse 
notation and language and refer to A J as an even integral lattice. We agree to 
identify A x with XA since locally everywhere these are identified (paralleling the 
fact that r(Xi, . . . ,X n ;X 2 J) = T(XX U . . . ,XX n ; J)). 
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Not every lattice is a free O-module, and given a lattice A, there are many ways 
to choose fractional ideals X, and vectors Xj so that A = T\ Say we 

have 

A = l lXl © • • • © X n x n = X[ yi © • • • © X' n y n , 

Vj = Y.i a ij x i- Then by 81:8 [6], X l ■ ■ -X n = X[ ■ ■ -X' n ■ det(a ij ). 

The Invariant Factor Theorem (81:11 [6]) says that given lattices A, fi on 
a (non-zero) space V, there are vectors x\,... ,x n G V and fractional ideals 
Ji, . . . , J n , Ai, ■ ■ ■ , A n so that 

A = X\X\ © • • • © X n x n , 

O = X\A\X\ © • • • © X n A n x n , 

and .4. j | .Ai+i (1 < i < n); the Ai are unique and called the invariant factors of Q 
in A. We use {A : 0} to refer to these invariant factors, and we write {A : 0} = 

(«4l , . . . , An) . 

When analyzing the action of Hecke operators on Fourier coefficients, we sum 
over lattices A where VA C O C P _1 A, ? a prime ideal and A a fixed reference 
lattice of rank n. By the Invariant Factor Theorem, we have sublattices A; so that 

A = A © Ai © A 2 , 

O = PA © Ai ®V~ 1 A 2 . 

So for instance, ro = rankAo is the multiplicity of V among the invariant factors 
{A : O}, denoted ro = mult{A:n}(^ 3 )- 

We will also need to consider (A J n Vt J )/V(A J + Vt J ) w Af /VA? . We will 
only be considering even integral A^ . Thus Q induces a quadratic form \aQ on 
Af /VA? defined by 

\aQ{x + VA X ) = \aQ{x) + V G 0/V 

where a G IK has been fixed so that aO-p = JO-p. Since Q(Ai) C 2J7" -1 , this gives 
us a quadratic form on the C/P-space A? /VA? . Note that the structure of the 
quadratic space A? /VA? is independent of the choice of a. 

Definition. Let / G Mk- Given any even integral positive semi-definite lattice 
A J with A = Xixi © • • • © X n x n , we set 

c(A J ) = c F (A J ) = Cf (T) ■ N(X 1 ■ ■■Xn) k N(J) nk ' 2 

where / G A / f/ c (r(Xi, . . . ,X n ; J)) is a representative of the component of F corre- 
sponding to cls(Ji • • -X n ), clx+J", and T = (B(x i7 Xj) ) . If k is odd, we assume A 
is also equipped with an orientation. 



8 



SUZANNE CAULK AND LYNNE H. WALLING 



Proposition 2.2. The "Fourier coefficient" c(A J ) is well-defined. 

Proof. First, suppose we also have A' = Tiyi © ■ • • © 1 n y n , and A = A'. Take 
M = (a^) so that 

(yi ■ ■ -Vn) = (xi ■ ..x n )M. 
Hence {B(y h yj)) = l MTM (recall T = (B(x h Xj))). Note that 

-^jVj ^ y ctjj-EjXj ^= A, 

i 

so aij G X{I~ . Also, since volA = volA', it follows that detM G O x . (Recall that 
if k is odd then A has an orientation, and detM must also be totally positive.) 

Thus ( M tM-i} e r ( J i 5 --- ,Zn,J), and so 

M \ 
t M~ 1 J ■ 

Hence 

f( T ) = 5^ c /( T ) e { Tr ) 

T 

= N(detM) k J2 c f( T ) e {™ T * M } 

T 

= J2c f (T)e{ 1 MTMt} 

T 

(recall that detM is a unit, and that if k is odd, a totally positive unit). Thus 
c f ( l MTM) = N(detM) k c f (T) = c f (T), and so 

c(A J ) = c f {T)N{X 1 ■ ■ ■l n ) k N(JT k t 2 

= c f ( t MTM)N(l 1 ■ ■■J n ) k N(J) nk / 2 = c(A' J ). 

Thus the definition of c(A^) is independent of the choice of basis relative to the 
coefficient ideals Xi, . . . ,X n and the scaling ideal J (so here Xi, . . . ,X n and J are 
fixed) . 

Next, fix J and suppose A = X\X\ © • • • © T n x n = X{y^ © • • • © 1' n y' n . Then by 
81:8 of [6], I[---T n E cls(Xi • • -X n ). Thus, as we have seen, M fc (T(Xi, . . . ,X n ; J)) ~ 
Mfc(r(X{, . . . , l' n ; J)) via an appropriate composition of the maps Ui, Vij; the action 
of this composition is given by 




/~/' = /i( M ,„-,) 
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where M e (X^X" 1 ) and (detM)X( • • • X' n = X x • • • X n . Also, 

f'(r) = N(detM) k f(Mr t M) = N(det M) k ^ c/(T)e{ 1 MTMt}, 

T 

so Cf (^MTM) = N(detM) k c f (T). Set (yi...y n ) = (%i ■ ■ ■ x n )M; thus £>yi©---© 
Cy n ~ l MTM. We claim that with A' = X' x y x © • • • ©X;?/ n , A = A. We know that 
writing M as (ct^), 

n 

= G (^)~ 1 ( :Z: i a; l © • • • © InXn) 

and so A' C A. Since norm(A) = (X 1 ■ • -X n ) 2 • detT = (X[ ■ ■ -X' n ) 2 • det(*MTM) = 
norm(A / ), we have A' = A. So 

c{A' J ) = c r { t MTM) N(X[ ■ ■ -X' n ) k N(J) nk / 2 

= c f (T) N(detM) k N(X[---X' n ) k N(J) nk / 2 

= c f (T) N{X 1 --.X n ) k N(J) nk / 2 

= c(A J ). 

Thus by our assumption, X[y\ © • • • © X' n y n = X[y' x © • • • © X' n y' n , which reduces 
the problem to the preceeding case. Hence our definition of c(A^) is independent 
of the choice of classes Xi, . . . , X n so that cls(Xi • • -X n ) is as prescribed (so here 
cls(Xi • • - In), J are fixed). 

Finally, suppose J 1 G clx+J. Thus J 1 = aX 2 J for some fractional ideal X and 
some a»0. Say A = X x x x © • • • ®X n x n . We have agreed previously to identify A x 
and XA, so that c/(A aX J ) = Cf(XA aJ ), whether we think of / as associated to 
T(Xi, . . . ,X n ; aX^J) or to T(XXi, . . . ,XX n ; aj) (remember, these are two names for 
the same group). So suppose J' = aj, a 0. We know M. k (T(Xi, . . . ,X n ; J)) ~ 
M k {T(X 1 ,... ,X n ;aJ)) via 

/->/' = /I ( a " 1J j)- 

Thus 

f\r) = N{a)- nk ' 2 f{a-\) = N(a)- nh / 2 £ Cf^e^Tr}. 

T 

Consequently c//(a _1 T) = N{a)- nk / 2 c f (T). Set A' =X x xx® ■ ■ ■ ®X n x n equipped 
with the quadratic form a~ x T (so (A') ^ is an integral lattice). Then we have 

c(A J ) = Cf (T) N(X 1 ■ ■ -X n ) k N(J) nk / 2 

= Cf^a-'T) N(X 1 ■ --X n ) k N(aJ) nk / 2 

= c((AY J ). 

Thus the definition of c(A J ) is also independent of the choice of the representative 
for clx+X □ 
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3. ElGENSPACES. 



The operators Vg(Q) are lifts of the Hilbert modular form operators that Eichler 
called V(Q~ 1 ) [3] and Shimura called S(Q) [7]. We now introduce another lift of 
these operators, and following Shimura [7] (where n = 1), we decompose M.^ into 
eigenspaces for these new operators. 

Definition. Let Q be a fractional ideal, and choose 



a b 
c d 



Q 



QL7 2 J~ 1 d 



Q 



-l 



with ad — be = 1. Set 



M 



(h-\ 



0i- 



V 



Oi-i 



h- 



d 



\ 



In- 1 



Then MT(X U . . . ,l n ; J)M~ X = T(J(, ... ,T n ;J) where 



J ■ = 



Thus S £ (Q) : M k (T(J u ... ,l n ;J)) -> A^ fc (r(J(,... ,X;;J r )) is an isomorphism 
where we define 

f\S e (Q) = f\M. 

Proposition 3.1. With Q,V fractional ideals and a G K x , St(Q) commutes with 
Ui(a), W(a), and V lJ {V). Further, Si(Q)Vij(Q) = Sj(Q) and U^a'^S^Q) = 
Si{aQ). 

Proof. Keeping in mind the various domains for different incarnations of our func- 
tions, it's easy to show Se(Q) commutes with Ui(a), W(a). 

To show Sg(Q), Vij(V) commute, it suffices to show Se(Q), Vi(V) commute (recall 
how Vij is defined). When £ ^ i, it is easy to see the matrices giving the actions 
of Si{Q), Vi(V) commute. As we explain below, we can reduce our attention to the 
case n = 2. 

Suppose £ = i or i + 1; for the sake of clarity, let us first look at the case where 
z = l,£=lor2. Then the action of each operator is given by a matrix of the form 



(A B ^ 
I 
C D 
\ Ij 
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where A, B,C,D are 2x2 matrices. Since the product of such matrices (and their 
inverses) is again of this form, it suffices to restrict our attention to the submatrices 
A B 
C D 

First consider n = 2, i = £ = 1. Choose 



a b 
c d 

so that ad — bc= 1. Choose 



A e 



Q QP 2 Z\Jd- x 



V' 1 VQI1I2 1 

v- x z~ x z 2 v 



so that det A = 1. (Note that these choices are possible, even when Q = V.) Then 



M = 



(a b \ 
1 
c d 
V 1/ 



gives the action of both 

Si(Q) : M k (T(Qli,l 2 ;J)) -> M k (T(li,l 2 ; J)) 

and 

Si{Q) : M k (T{VQZi, V~ X Z 2 - J)) -> M k (T{Vli, V~ X Z 2 - J)). 
Similarly, N = tj^-i ^ gi ves the action of both 



Vi(Q) :^ fc (r(QJ 1 ,J 2 ; l 7)) ^ M k {T{VQX x ,V- x Z 2 -J)) 



and 



Vi(Q) : jWfc(r(Ji, J 2 ; J")) Mk(^(VZ 1 ,V~ 1 Z 2 ;J)). 

A simple (but tiresome) check shows MNM^N -1 e r(QJi, J 2 ; J"), which 
implies that <Si(Q),Vi(P) commute when n = 2. For general n, we have 
£i(Q)Vi(P)5'i(Q _1 )Vi('P _1 ) represented by a matrix of the form 



M' 




where 



(J, ^ e r(QZuZ 2 ;J). Thus M' e r(QJ 1 ,J 2 ,... ,J n ;J), 



and 



C D 

Si(Q), Vi(V) commute for general n. Similarly, S 2 (Q), Vi(V) commute for n = 2, 
and thus for general n. 
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For general n, £, i with £ = i or the matrices giving the action of Se(Q), Vi(V) 
are of the form 

(h-i 0,_i \ 

A B 
I 
Oi-i Ii-i 

C D 
V ij 

where A, B,C, D are 2x2 matrices. So again the problem reduces to showing 
Si(Q), Vi{V) (£ = i or i + 1) commute when n = 2 (which we have done). 

To see that Si(Q)Vij(Q) = Sj(Q), it again suffices to consider j = i + 1. First, 
fix a group V = . . . ,I n ; J\ From our definitions, we have a product of three 
matrices giving the action of Si(Q)Vi(Q)Si+i(Q~ 1 ) on M. k {Y). One easily verifies 
that the conditions on these matrices ensure the product for Si(Q)Vi(Q)S i+ i(Q~ 1 ) 
lies in T. Finally, one verifies U i (a~ 1 )S i (Q) = Si(aQ) by matrix multiplication. □ 

This shows that the Sg(Q) act on Mk, where, for F G Mk, F ~ (. . . , f t , . . .), 
F\St(Q) ~ (••• ,fi\S e (Q),...). It also shows that Si(Q), Sj(Q) are equivalent on 
M.ki and so we simply refer to this operator on Ai k as S(Q). Furthermore, on Ai k , 
S(Q) = S(aQ) for all a G 1K X ; since we also know S(*) is multiplicative, the map 
clsX I— > S(I) gives a group action of the ideal class group on M. k - 

Proposition 3.2. M. k = ® x M. k (x) where x varies over all characters of the ideal 
class group, and 

M k ( X ) = {FeM k : F\S(Q) = X (Q)F for all Q }. 

Proof. First notice that M.k{x) ^ ■Mfc(V') = {0} if x V'- To prove this take 
F G M k (x) H -M fc (V0- Then X (Q)F = F\S{Q) = ip(Q)F for all Q. Since x ^ V> 
there is a Q such that x(Q) ^ V'(Q)- Therefore F = 0. 

For ^ an ideal class character, let = — ^^^(T) F\S(T) where h is the class 

clsX 

number of K. Note that 

1 / v^— / x\ , , s v^— x (h if clsX = clsC, 

E G* = 5 I E *W TO - F -ce £ *(2) = 

i/j clsi, \ W / W 



Thus F = £^6^. 
Next notice that 



G*|S(e) = £5>p) F|5(2)|5(Q) 

clsX 



clsX 



clsX 

= V(C) 
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Thus e Mk(ip), and hence F E ® x M k (x)- □ 

4. Hecke operators. 

We begin by defining the Hecke operators. Then we show that they act on each 
Aik(x)- After this, we describe how to find a set of coset representatives giving 
the action of the operators. Finally, in the next section we analyze the action of 
the operators on Fourier coefficients attached to even integral lattices, proving our 
main theorem. 

Definition. Let V a prime ideal; set V = r(Ti, . . . ,T n ; J) and V = 
r(Ji, . . . ,l n ;VJ). We define the Hecke operator T(V) : M k (T') -> M k {T) by 



F\T(V) = N{V) n ^ k ~ n ~ 1)/2 ^ F \l 



7 



where 7 runs over a complete set of coset representatives for (r ; n r)\r. Note that 

(VI \ 

V is the formal conjugate of V by the matrix 8 = ( n j )• When K = Q, we 
define T(p) on M k (T) by 

f\T{p)=p n{k - n - 1)/2 Y J f\ 8 ~ 1 i 

7 

where 7 runs over a complete set of coset representatives for (r'nr)\r, r' = 5r5 _1 , 
^ _ ^pl ^ (This normalization of T(p) is standard, and as with the standard 

normalization of the degree 1 Hecke operator T(p), the purpose of the normalization 
is to force the coefficient to be 1 on the "lead" term in the expression for the Ath 
coefficient of F\T(p).) 

Now fix 1 < j < n; let rj = T(Vli, ... , VTj,I j+ i, . . . ,X„; J"). We define the 
Hecke operators Tj(V 2 ) : M k {^'j) -> -M fc (r) by 

F|T J (P 2 )=^F| 7 

7 

where 7 runs over a complete set of coset representatives for (r'- fl r)\r. 

Note that r' is the formal conjugate of V by diag("P2j, I n -j,V~ x Ij, I n -j)- When 
K = Q, V = pZ, we define Tj(V 2 ) = T^p 2 ) on A^ fc (r) by 

/l^(^ 2 ) = ^/|5- 1 7 

7 

where 5 = diag(plj, I n -j, ^Ij,I n -j), V = 5r5 _1 , and 7 runs over a complete set of 
coset representatives for (r ; fl T)\r. (We introduce a normalization later.) 
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Proposition 4.1. The operators T(V), T^V 2 ) commute with Ui(a), W(p), V U (Q), 
and Si(Q) where a, (3 G K x with (3^0, and Q is a fractional ideal. Thus T(V), 
Tj(V 2 ) act on M.kix) ( as defined in Proposition 3.2). 

Proof. To show Tj(V 2 ) commutes with the Vu(Q) : it suffices to show it commutes 
with Vi(Q). Take 

Or 1 Ta~ x QV~ 



A G 



with det A = 1, and set 



T' = 



VZ t i£i<j, 
X, otherwise. 



Then with 



M 



I-n—i — l 



t A-l 



\ 



-J 



M gives the action of V l {Q) : M k (Y) -> M^M^YM) and of V l (Q) : M k (Y') -> 
Mk(M~ l T' M.) (note that these are the appropriate codomains). Now let {7} 
be a complete set of coset representatives for (Y' n T)\r. Thus {M~ l ^M} is a 
complete set of coset representatives for (M _1 r'M n M _1 rM)\M _1 rM. Hence 
for/GM fc (r'), 

f\T J (V 2 )V t (Q)=J2fh\M 

7 

= j2f\ M \ M ~ 1 i M 

= m(Q)T J (V 2 ). 

Similarly, to show Tj(V 2 ) commutes with Si(Q), choose 

Q Q -i X 2 V 2 Jd -i 



j 

a b 
c d 



QI~ 2 J- x d 



Or 1 



so that ad — bc = l. Then 



d 



lifts to a matrix M so that M gives the action of 



Si(Q) : M k (Y) -> M^M-^M) and of S t (Q) : -M fc (r') -> ^(M^r'Af). Thus 

/|T J (P 2 )^(Q) = ^/| 7 |M 

7 

7 

= /|^(Q)T i (P 2 ). 

Similar but simpler arguments show that Tj(V 2 ) commutes with Ui(ct), W(f3), and 
that T(V) commutes with Vi(Q),Si(Q), Ui(a),W(/3). □ 
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Proposition 4.2. For f G Mk(T'), we have 

n,Ai,y x ; 



Here O varies over all lattices such that VA CflC V~ 1 A, Ai varies over all codi- 
mension n—j subspaces ofQC\A/V(0,+A), C = C(fi, Ai). Withro = mult^.^yiV), 
mi = mult{\-xi}(0) , r\ = mi — n + j, fi G V~ x — O fixed, 



ro+ri 



sW{si)=[ n SiCP))[ n ^ 

\i=r + l / \i=r +r 1 + l 



fW W 2 W 3 

Y = 



f W 2 fiWt 




Vw 3 

with Y G (Iiljj'd~ 1 ), Wo varying modulo V 2 , Wi,W 2 ,W 3 varying modulo V with 
V not dividing det W\. (Recall that d is the different ofK.) Here Wq is ro x ro and 
symmetric, W\ is r\ x r\ and symmetric, W 2 is ro xri, W 3 is ro x (n — j). 

Proof. Let P be a prime ideal, and fix j, 1 < j < n. We essentially follow the 
algorithm presented in [5] to find a set of coset representatives giving the action of 

T 3 \V 2 ) :M k {T{VX u ... ,Vlj,l j+ i,... ,X n ;J)) ^M k {T{X u ... ,X n ;J)). 

For convenience, we will take Xj = O for 1 < i < j, Xj = O for j < i < n; also, we 
take X = I n and Jd~ x to be integral ideals relatively prime to V (recall that the 
equivalence class of Mh(T) is determined by clsX, cls + J, and d is the different of K). 
Note that this allows us to choose \i relatively prime to X\ ■ • - X n Jd~ x = XJd~ x . 

Choose M G T(0, ... ,0,X;J), and let M 3 = (A\B) denote the top j rows of 
M with A, B j x n matrices. Let A = Opxi © • • • © Opx n be a reference lattice. 

Step 1. Let 

fi = ker(A -> A(A) mod VO v ) 

where A = (a\ ■ ■ ■ a n ) and A — > A(A) mod "PO-p denotes the map that takes Xi to Hi 
(which is a 1 x j matrix with entries in Op/VOp). Note that the ro = rankpA(A) 
is at most j since A is a j x n matrix. 

We claim there is a matrix ^ t C~ 1 ) e r(0, . . . , (9,X; J") such that 
Oo = A^of P/ro r and f C t _ x ) = with a' 1; . . . , a' n = 



1 n—ro J V / 

O(modP). 

First, write A = (ai . . . a n ) and consider the rank modulo VOp of (ai . . . a n ). 
Let i?i be an (n — 1) x (n — 1) invertible matrix (i.e. a change of basis matrix) so 
that 

(ai . ..a n )Ei = (a[ . . .a' n ) 



16 



SUZANNE CAULK AND LYNNE H. WALLING 



with a[, . . . ,a' rQ linearly independent modulo VO-p and a' ro+1 = 



= a 



n-l — 




(mod VO v ). Note that G 1 = \ l E^ x \ G F(0,... ,0,1;J). If a n is 

f I * 

in the span modulo VO-p of a[ , . . . , a' ro , then there is a matrix E2 = ( , 

such that (a^ . . . a^ l _ 1 a n )i?2 = (a' 1 ...a' n ) where a' n = (mod note that 

G 2 = ^ 2 er(O r .. , C, J; J") and we take C to be GiG 2 . Ifr = n-1 

then we are now done, regardless of whether a n is in the span (modulo VO-p) of 
a[ , . . . , a' ro . 

So suppose ro < n — 1 (and thus a^_ x = (mod VO-p)) and a n is not in the 
span (modulo VOp) of a[, . . . , a' r() _ 1 . 

Choose rj G G X such that 77 = p = 1 (mod P) and choose v <E V 

such that {y,r]p) = 1. Thus there are a, /3 G (9 so that az/ — prj(3 = 1. 

Then with E 3 = ^ a r//3 j , ( a ; . . . a'^a^Es = (a[ . . . a^_ 2 a^_ 1 a^) with 

<_! = a n (mod P), a" = (mod P). Note that G 3 = ^ 3 
r(C, ... ,0,T;J). Let £4 be the permutation matrix that permutes columns ro 
and n-l; then G 4 = t E -i^j e T(C, . . . ,0,1;J) and (ai . . . a n )GiG 3 G'4 = 

(a' x . . . a^. o _ 1 a n ... 0) (mod V). Hence in this case we take C = G1G3G4. 

Thus there is a matrix Co and integer ro such that t^-i ^ e 

r(<9, ... ,0,X;J) and fi = AC ( VIr ° J. Then with renewed nota- 

tion, Mj tQ-i^j nas the form (a\, . . . , a n \b\, . . . ,b n ) = (AqAi\B), Ai = 

(mod 

Note that while Cq is not uniquely determined, Qq is. 

Step 2. First note that Lemma 7.2 of [5] easily generalizes to number fields, 
where we "permute" bt and b n as we "permuted" a' rQ and a n in the preceeding 
paragraph. Thus with 

Mj (^ Co tc _^ = (A\B) = (ai,... ,a n \h,... ,b n ), 

where 61,... , b ro are in the span mod VOp of at,... ,a ro , and the rank mod 
VO-p of (ai,... , a ro , 6 ro+ i, . . . ,b n ) is j. Thus for some C = (^ r ° with 

( C tc _i) G r(0,... J), we have ^ ^ ) = with 

= for % < ro, and j the rank mod VOp of (at, . . . , a ro , 6j. +i, • • • ,b'A. We 
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want to accomplish the above rearrangement, as well as replacing bj +1 , ... ,b' n with 
vectors in the span mod VO-p of a±, . . . , a ro ; we want to identify these modifications 
with a uniquely determined lattice. 

From Step 1 we have A = A © Ai, fi = PA ffi Ai with Ai uniquely determined 
modulo PA. This corresponds to a splitting A # = A © A[ of the (formal) dual of 
A, where Aq is orthogonal to Ai and A^ is orthogonal to A . (So 

A # = O vVl © • • • © O v y n 

and the basis {y±, . . . , y n } is dual to {xi, . . . , x n }.) 

Let V be the O-p/PO-p-space consisting of all j x 1 matrices, and let U be the 
subspace spanned by a\ , . . . , a ro . Let 

Q[ = ker(A # -> A # (S) mod P0 P -> V/f7), 

where A# — > A#(B) corresponds to i— > 6j, and the map into V/U is the canonical 

E 

that (° t c -i^j e r(C, ... ,0,1;J) and t^-i) = (^'l 5 ') with 

6q = 6o> • • • j ^r = ^r ! ^j+i? m the span mod VO-p of ao, . . . , a ro . Also, setting 

(Ir 

n[ = A* 'Cf 1 VIj- ro 

\ In- 3 

A* = A* t C^ 1 = A' Q ® A' 2 © A3 with rankA 2 = j - r and A' © A 3 uniquely 
determined modulo VA#. Correspondingly, 

Slo = Ad ( VIro 



predion .nap. Thus a, in Step !, we can «n d a matr ix C = , ^ .„ 



In—r 

Note that A = Ad = A © A 2 © A 3 . Set 

fii = Ad I 7^ I = VA © A 2 © PA 3 . 

VI n -j 

Since A © A 3 are uniquely determined modulo PA#, A 2 is uniquely determined 
modulo PA. 
Step 3. Write 

Mj ( Cl t(J _A = (a l ---a n \b l ---b n ) = (AoAtAslB^Bs) 
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where j is the rank modulo VO-p of (A , B{), A\, A 3 = (mod VO-p), and B , B 3 
are in the (column) span modulo VOp of Aq. We want to modify A\ to be of the 
form (A[, A' 2 ) where A' 2 = (mod P 2 C P ). Recall that we have 

A = A ©A 2 ©A 3 , fii =PA © A 2 ©PA 3 , 

with rankA = r , rankA 2 = j — tq. Renewing our notation, let {x\, . . . , x n ) be a 
basis corresponding to this decompositon of A. 

Recall we have fixed \i G V~ x — O so that \i is relatively prime to XJd~ x ; set 

Vtt 2 = ker(Oi -> Oi(^A) mod 

where fii — > denotes the map taking to fia^ so /iOi(A) mod PO-p 

is spanned by a\, . . . ,a ro , fm ro +i, . . . , [Mj with a±, . . . ,a ro linearly independent 
modulo VO v (recall a { ~ (mod for i > j). Thus 

Pfi 2 = ^ 2 A © PAi © A 2 © PA 3 



where A 2 is uniquely determined modulo VO.\. As in the previous steps, we can 




such that I C t ! ) G T(C, ... ,0,X;J) 



/-P 2 I ro 



VI ri 

VI n - 3 



Correspondingly, tQ-i^ = {AqA'^A^BqB'^B^) with A[ = 

(mod VO-p), A 2 = (mod V 2 Op), and ai, . . . , a ro , A"4 +i' • • • , [ia' ro+ri lin- 
early independent modulo VOp where v\ = rankAi ( and so A'± is j x n). Let 
C(fi,Ai) = C Ci. 

Step 4. Write M j ^ ^ = (A , A l5 A 2 , A 3 |B , 5 1? B 2 , B 3 ). So A 1; A 3 = 

(mod VO-p), A 2 = (mod V 2 Op), and the columns of (A ,/iAi) are linearly 
independent modulo VOp. Also, B , B 3 are in the column span modulo VO-p of 

Since So, -Bi are in spanpA , we can solve 

A Y ' = -B (mod VO P ), A Y 3 = -B 3 (mod VOp). 

Note that as B 1 A is symmetric and A\, A 2 , A 3 = (mod VOp), Bq 1 Aq is sym- 
metric modulo VOp. Also, since Aq has full rank modulo V, there is some ma- 

A'\ ,„... /S' 



trix £■ G GLj(O-p) such that i?A = ( q j • Writing ES = ( B „ ) , we see 
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B" = (mod VOp) since E(B l A ) l E is symmetric modulo VOp. Thus Y ' 
is the unique solution modulo V to A'Y ' = —B' (mod VOp); since (A') -1 B' is 
symmetric modulo VOp, we can choose Y ' to be symmetric. 
Let 



(Aq, Ai, A 2 , A 3 \B' , Bi, B 2 , B' 3 ) 



I 1 



(A ,A 1 ,A 2 ,A 3 \B ,B 1 ,B 2 ,B 3 ) 



Y Y 3 \ 





I *la 



V 



I J 



B' , B' 3 = (mod VO-p). Then just as we argued about Yq, there is a unique modulo 
VO-p symmetric solution Y' to 

(A ,^A 1 )Y' = -(^(B + A 3 t Y 3 ),B 1 ) (mod VO v ). 



Decompose Y' as 



Y" Y 

t ^ v 2 ) ; choose 5 G P so that S/j, = 1 (mod "P) and set 



>2 n 



y = y ' + «o"- 

Note that since rankp(Ao, B{) 



tq + r±, we have rankp(i?i + AqYq 



rankpi?! = T\. Since — /MiY - ! = B 1 +A Y 2 (mod VOp), we must have det Yi G C^. 

to be a symmetric ma- 



Take Y = I J 
*W 3 



w o w 3 \ Z r w * 

' l W 2 ixW 



l 



7 

Vw 3 J 

trix in (l.ljjd- 1 ) with W = F (mod V 2 O v ) and = (mod VOp) for 
i = l,2,3. 
Then 



(A , A x , A 2 , A 3 \B , B u B 2 , B 3 ) 



I Y 
I 



(Ao, Ai, A 2 , A 3 \Bq, B[, B' 2 , B 3 ) 



with B' ' = (mod V 2 O v ), B' 2 = B 2 (mod V 2 O v ), B[, B' 3 = (mod VOp). 
Let C = C(Q, Ai). Also, identifying Si(V) with a matrix giving its action, let 



n 

j=r + l 

We see that, with renewed notation, 
C 



n s ^ 

^j=r +ri + l 



(° t c -^j{^ Y I ^jS- 1 (n) = (A ,A 1 ,A 2 ,A 3 \B ,B 1 ,B 2 ,B 3 ) 
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with A 3 , B 3 = (mod VO r ), B , B u B 2 = (V 2 O v ). 

However, while the matrices for Si(V~ 2 ) lie in T, the matrices for S^V -1 ) do 
not. We remedy this as follows. 

For r < i < r + ri, choose a % G V' 1 , fa G Vlfjd' 1 , 7; G V~ l l~ 2 J- l d so 
that — fii^i = 1 and for any prime Q ^ V dividing 5, Q does not divide fa. 
(Recall that our choice of 5 ensures 5 G V — V 2 .) Set a = diag(. . . , c^, . . . ), an 
ri x n matrix; define fa 7 in an analogous fashion. So 



r 



\ 



a 







V 











r J ro 

7 51 
~ J/ 



gives the action of U?=r +i S l {V~ 1 ). 
Now consider 



I -/iWA fa §_ 
I { 7 SI 



7 — 5/ J ( mod n 



We find that I — ^ J is a coprime symmetric right-hand pair for 

r(2 ro+1 , . . . , T ro+r i ; J) (2r[ x 2r^ matrices). Thus by Lemma 6.1, there exist ma- 
trices U, V so that 



(U (3 + dW 1 \ . 



Hence 



0r„ \ 

U (3-Wt 
I ~ 



V 



J r 1 r 

V SI 



and with Y' 



W x W 2 



l W 2 I and 



C 



I -Y' 
I 



x- 1 ( n s ^ v ~ 2 ) 
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we have MjN~ x = (A , A u A 2 , A 3 \B , B u B 2 , B 3 ) with A 3 , B 3 = (mod VO v ), 
So, Bi, B 2 = (mod V 2 0-p). Thus by an easy generalization of Lemma 7.1 of [5], 
MN' 1 G T n r. Also, since 



5/ 

~1 
we have 



V 51 



V I ) E ^^0+1' 



f'\N = f'\S(Q) 
Proposition 4.3. For f G M k (T(li 



I -Y 
I 



C 



-l 



. □ 



T-n'i VJ)), we have 
\ 

f'\T(V) = N(V) n(k - n - 1)/2 J2 f'\ s (ty 



Q,Y 



(I Y 

I 
I 

ij 



c- 1 



and Yq G {Xil^Jd x ) varies over symmetric r x r matrices modulo V and C varies 
as in Proposition 4-2. Here r = mult^.Qy(V) and S(fl) = Y\7=r+i ^liV). 

Proof. To find coset representatives for T(V), take M G F(Ii,... ,X n ; J~); write 
M = (c ^)- Let0 = ker ( A ^ A ( A ) (mod V)), and choose C = C(Q) so 

that O = AC ( VIr ). Thus (A\B)C= (AqA^BqB^, A x = (mod VO v ), 

\ J-n—r J 

rankpAo = r where Aq is n x r, and So G span-pAo. Choose symmetric Yb with i, £■ 

( Y 

entry in T^Jd' 1 such that A Y = B (mod Then with Y - ' ' ' ; 



(A\B) 



C 



-r 



S" 1 ^) = where S{tt) = flLr+i and 



so that the action of S(Q) is given by 

f 1 



I -Y 
I 



(I x 

a (3 
I 
V 7 SI J 



a 

1 
-Y 



V 



(n 




r) x 


r 




\ 


I r 








51 J 


\ 




f 1 










J 




V 



. Then 







\ 



a + /i7 (3 + 51 
I 

51 J 



7 



Here /3 = (mod PCp), rank P (/3 + //M") = n - r. Thus ^ 



(3-(jl6I 



51 



is a sym- 



metric coprime right-hand pair for F(X r+ i, . . . ,I n ; J\ hence there are U, V so that 
I 1 \ 



G T{l u ... ,X n ;J). Since 



U f3 + fi5I 
I 
\ V 51 J 

r(J r+ i, . . . ,X n ;VJ) we get the result as claimed. □ 



U (3 + ^51 
V 51 



51 -p 
~1 QL 
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5. Evaluating the action of the Hecke operators. 

When evaluating the action of the operators Tj(V 2 ), we encounter incomplete 
character sums. To complete these, we define modified operators as follows. 

Definition. For V a prime ideal and 1 < j < n, define 

f J (V 2 )=N(Vy( k - n - V J2 fi(n-£,j-e)S e+1 (V)-'-S j (V)T e (V 2 ). 

0<l<j 

We will also need the following rather technical result. 

Proposition 5.1. Let T\ be a symmetric v\ x v\ matrix whose i,£-entry lies in 
(I ro+ iT ro+ eJ')~ 1 , and whose ith diagonal entry lies in 2X~ Q+i J~ 1 ; fix [i G V~ x — O. 
With W varying over all symmetric r\ x r\ matrices modulo V with i,£-entry in 



W 0<m<r; A,U 



where for each m, A varies over dimension m subspaces of A\, A ~ Ta (mod V), 
and U varies over all m x m symmetric matrices modulo V . 

Proof. For a moment, let's fix W. Since W is symmetric, we can view it as 
the matrix of a quadratic form on an r[ dimensional O/V space V = L/VL, 

L = (X~ o+1 yi © • • • © l ro+ri y ri ) . (When V is dyadic, let W define an integral 
quadratic form on 

L = (O r X-\ iyi © • • • © O v l; Q \ ri y ri ) J - ld , 

and let V = L/VL, a quadratic space over O v /VO v m O/V. We use §93 of [6] 
to understand the structure of L and thereby of V.) The radical of this space is 
uniquely defined, so for some G G GL ri (0), 



t G~ 1 WG~ 1 = ) ( mod ' P ) 



where U is m x m with rank-pU = m. (So VG 1 = J © radV* where J is a regular 
space whose isometry class is uniquely determined by V, and J ~ U.) 
So 

e{aTi W} = e{«Ti *G ^ U Q ^ G} 
= e{a(GT 1 t G)( U Q )} 



= e{aSU} 



where GT\ l G = ( * }, S an m x m matrix. Here we take A to be a rank 



n lattice as in the previous section, and we equip A with a quadratic form such 
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that A ~ T; thus with A x as in the previous section, we have A x ~ Ti, and 
A = Ai t G ^ ^™ ^ ~ S. So S corresponds to an m-dimensional subspace A of 

the O/P-space Ai. Thus each W gives rise to (at least one) pair (A,U), A an 
m-dimenstional subspace of Ai , U an m x m integral symmetric matrix of rank m 
modulo V. 

With Ti still fixed, fix m, < m < r±. We now define a map <p from all pairs 
(A, U) as above to symmetric v\ x n matrices W. Here A is an m-dimensional 
subspace of A x , and U is an integral symmetric m x m matrix with rankp£7 = m. 

For each such A we fix some G = G A G GL ri (O) so that A = A~i *G ^ 7 ™ ^ . We 

define v?( A, U) = l G G. 

We first show that the image of <p consists of all symmetric r\ x r\ matrices W 
modulo V with rank-pW = m. Then we show that ip is injective. 
As shown above, given any W in the codomain of cp, 

W= l G Q ^JG (mod V) 

where U is m x m, m = rankpt/, and G G GL ri (O). 

Take A = Ai *G (^™^ . So A is an m-dimensional subspace of Ai, and thus 

t G (^Q^j an d t( ^A (^Q^j eac ^ ma P a basis f° r to a basis for A. Hence with 

(xi,... ,x r J a basis for Ai, (xi,... ,x ri ) *G = (yi,... ,y ri ), (xi,... ,x ri ) *G A = 
(zi, . . . , z ri ), we must have (yi, . . . , y m ) = (zi, . . . , z m ) *C (mod V) for some C 



G 



GL m (0). Thus (xi,... , x r J *G = (xi,... , x ri ) f G A ( q j, meaning *G 



Hence modulo T 3 , 







Thus <p is surjective. 

Now we show cp is injective. Say 

W = <p(A h Ui) = <p(A 2 , U 2 ) (mod P). 

Thus with Gi = G Ai , we have 

try I Ui \ r< _ try [ U2 



W= t G 1 [" L Q \G 1 = t G 2 i^ z )G 2 (modP). 
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So with G = G 2 G^\ 

Since the columns of Ui are linearly independent modulo V, we must have G = 

(» °) Nn 

Now we compare A 1? A 2 ; we will find that Ai = A 2 , so Gi = G2 and hence 
C/i = U2 (mod P). With notation as before, we have 

(yi, • • • ,2/m) = (a;i, • • • ,x ri ) *Gi 

(zi, . . . ,z m ) = (xi, . . . ,x r j) *G 2 

= (xi,... ,x ri ) *Gi *G 

Thus given our knowledge of G, we see that (y±, . . . , y m ) = (z±, . . . , z m ) *C, and 
hence Ai = A 2 . Thus G\ = G 2 , and consequently U\ = U 2 (mod V). Therefore (p 
is injective. □ 

We can now prove our main result. In the remark following the proof we demon- 
strate how to compute the geometric term ct,-(f2, A). 

Theorem 5.2. Let F e •Mft(x) where x is a character of the ideal class group and 
■M-kix) i s as defined in Proposition 3.2. 

(1) The A J th coefficient of F\fj(V 2 ) is 

N(V) Ei(A ' n) x(V) e ^ A ' n) aj(Q,A)c F (Q J ) 

VA<zncr- 1 A 

where Ej(A, O) = k(r 2 - r + j) + r (r + mi + 1) + n(ri + l)/2 - j(n + 1), 
ej(A, O) = 2r 2 + ri = r 2 — r + j, and aj(Q, A) is i/ie number of totally isotropic 
codimension n — j subspaces o/flfl A/"P(fi + A), f/ere ro = rnult^.^iV) , 
mi = mult{ A .£iy(0), r 1 = m 1 - n+j, and r 2 = mult^.^V' 1 ) . 

(2) The A J th coefficient of F\T(V) is 

iV(P)^' A) x(P) n " r c F (0^" 1 ) 

•pAcncA 

where r = mult^.Qj(O) and E(Q, A) = k(n — r) + r(r + l)/2 — n(n + l)/2. 
Proof. Take fractional ideals Ji, . . . ,T n ,J and /' G .Mfc(r(X(, . . . ,l' n ;J)), 

x rpx, i£i < i, 

\li if i > j. 






HECKE OPERATORS ON HILBERT-SIEGEL MODULAR FORMS 



25 



In the preceeding proposition, consider the subsum where we fix a choice of O: 

Ai,y V / V / AljY V / V / 

where /" is the component of F corresponding to the group r(Jf , . . . ,Z^'; J"), 

PJ, if 1 < % < r , 
1" = { V~ l li ifr + n<i<j, 
Xi otherwise. 

Set mi = 7*i + n — j. Expanding /" as a Fourier series supported on even T e 
((I^'l'/J)- 1 ), we find that for fixed A x , 



t y 
= ^ C/ »(T)e{TC- 1 r*C- 1 } 

T 

^ e{T W } e{//lWi} e{T 2 W 2 } e{T 3 W 3 } 

/T T 2 * T 3 \ 

Ti * * 

where T = . T and Wo are symmetric r x tq matrices with 

* * * * 

\*T 3 * * * / 

T even, the z,£-entry of W$ in X(I^Jd~ x . Thus the sum on Wo (T fixed) is a 
complete character sum, yielding N(V) r ° (r ° +1) if T = (mod V), and otherwise. 
Similarly, the sums on W 2l Ws are complete character sums. So 

Ye{TY} = I N ^) ro(ro+rni+1) E Wl efciZWi} if T e ((T.W)" 1 ), 
y 1 1 otherwise. 

With (B(xi, xi)) = l C~ l TC~ l , take A = Zixi © • • • © T n £ n - Then the sum on 
TTo, TT2, W3 is nonzero if and only if A J is even integral. 

Let (yi .. . y n ) = (xi . . . x n )C and set O = X"yi©- • -©Z^/ n . Then ye)) = T 

and c r {T)N{X'{-- ■X' 1 [) k N{J) nk ' 2 = c F (n J ). 

Note that when cj|5^-p)(T) is contributing to the A^th coefficient of f\Tj(V 2 ) G 
M k (T(Ii,... ,X n ;J)), h gets normalized by N{X 1 . . .X n ) k N( J) nk / 2 ; when it is 
determining a coefficient of f\S l (V) E M k (T'), V ~ r(p-%,T 2 , . . . ,X n ;J), it 
gets normalized by N(V)- £k N(X 1 . . .X n ) k N(J) nk / 2 . 
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So the contribution from /" to the A^th Fourier coefficient of f'\Tj(V 2 ) is 

N ^ V y{k-n-l)+r a {r n+ ra 1 + l) x ^2r 2+ r' 1 N (V) k ( 2r ^ ) C F (Q J ) e {/" T Al W l } 

n,Ai Wi 

where O varies subject to PA C O C V~ X A, mult {A:fi} (p- 1 ) = r , mult {A:n} (P) = 
r 2 , Ai a codimension n— j subspace of Oi ~ fin A/P(Q + A), and W\ varies modulo 
V with z, Gentry in X{XiJd~ x , V /det Wi. Here T Al = (5(x ro+ i, x ro+ ^)) is ri x ri 
where Ai = X ro+ ix ro+ i © • • • ©X ro+ri x ro + ri . If T{ is also a matrix associated to 
Ai then there is a change of basis matrix G whose i, £-entry lies in X{XJ X so that 
t GT[G = T Al . Thus einTiWx} = e{^T Al (GWi *(?)}; as Wi varies over invertible 
matrices modulo "P, so does GW\ l G. Thus the sum on Wi is independent of the 
choice of matrix associated to Ai. 

We complete the character sum on W x by replacing Tj(V 2 ) with Tj(P 2 ), then 
apply Proposition 5.1, where we consider *Y1 W e{aTiW} with W varying over all 
symmetric r\ x r\ integral matrices modulo V. 

Notice that for < £ < j, 

S t+1 (V) ■ ■ ■ Sj{V)T t {V 2 ) : M k (T(I[, ...,I' n ; J)) - M k {V{X u . . . ,X n ; J). 

Also notice that the number of dimension n = mi —n+j lattices Ai containing some 
dimension mi — n + £ lattice A is the number of ways to extend A to a j-dimensional 
subspace of fii (where fi = fi © fii © ^2, A = Pfi © fii © "P _1 2 ). Extending 
A is equivalent to choosing a j — £ dimensional subspace of an n — £ space (here 
dimOi = mi). So the number of Ai containing A is f3(n — £,j — £) = f3-p(n — £, j —£). 

Note that the coefficient of f'\fj(V 2 ) associated to A J = (X x x\ © ••• © 
X n x n )^ carries a normalizing factor of N(Xi ■ ■ ■X n ) k N(J') nk / 2 , while the coeffi- 
cient of f'\SW(V) associated to fi^ = (X'/yi © • • ■ © I"y n )^ carries a factor of 
AT(p) fc ( r o-r 2 )jv(J 1 ■ • -X n ) k N(J) nk / 2 . Hence, contributing to c(A J ) we have 

j^^^k(r2-r +j)+ro(ro+m 1 +l)+r 1 (r 1 +l)/2-j(n+l)^^p^2r2-r c(fi^) 

Ai 

where A? varies over all totally isotropic codimension n — j sublattices of A^ fl 
VL J /V(A J + Q J ). Summing over all fi, VA CflC V~ x A, yields (1). 

The proof of (2) is quite similar to the proof of (1), but much simpler, and so 
we leave it to the reader. □ 

Remark. As discussed above Proposition 2.2, A? /VA? is a quadratic space over 
O/V. By §42 of [6] (for results about quadratic spaces over fields of characteristic 
2, see, for example, §5 of [9]), A? /VA? = R JL W JL H* where i? = radAf/PAf, 

is anisotropic, and H ~ ^ J ^ denotes a hyperbolic plane. With U = R _L W, 

Lemma 1.6 of [8] and Lemma 4.1 of [9] tell us that the number of ^-dimensional 
totally isotropic subspaces of A? /VA? is 



(A?/VA?) = q {t ~ a)(e ~ a) S(d + t-£ + a + l, a)p(t, a)<p^ a (U) 
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where q = N(V), d = dimt/, 8(m,r) = UZo(Q m ~ l + 1), P(™,r) = YlZoiv™^ ~ 
l)/(q r ~ l — 1), and < a < £. (Note that Lemma 1.6 of [8] is proved for a qua- 
dratic space over Z/pZ, but the argument is valid over all finite fields. When the 
characteristic is 2, we replace Q by we present a full discussion of this case 
in §5 of [9].) Also, since U = R _L W with R totally isotropic and W anisotropic, 
any totally isotropic subspace of U is a subspace of R. Thus (pe_ a (U) = /3(r, £ — a) 
where r = dimi?. (So d = dim £7 = r + dimW, and since W is anisotropic, dimVF 
is 0, 1, or 2 by 62:1b of [6] for q odd, and by, e.g., Proposition 5.1 of [9] for q even.) 
Hence 

aj(fl, A) = q {t ~ a)(e ~ a) S(d + t + a+l, a)/3(t, a)/3(r, I - a) 

a 

where < a < £, £ = r\ — n + j. 



6. Lemma on symmetric coprime pairs. 

Definition. We say a pair of matrices (C, D) is a symmetric coprime lower pair 
forr(J 1 ,...,J n ; t 7)if: 

(a) C f D is symmetric; 

(b) C e J-'d (l- 1 !- 1 ), D e (T-%-); 

(c) for all prime ideals V, 



Ai 



ranki 



(C,D) 



V 



a\ 1 1 



\ 



= n 



where aO v = Jd~ x O v , \O v = X l O v . 

Since T(X U . . . ,l n ; J) « T(C, ... ,0,1;J) when J G cls(2i • • • J n ), the follow- 
ing technical lemma focuses on T(0, ... ,0,X;J^), but the conclusion (c) is valid 
for r(Zi,... ,x„ ;t 7). 

Corresponding definitions and results hold for symmetric coprime upper, right- 
hand, and left-hand pairs for r(T 1? . . . , J n ; J). 

Lemma 6.1. LetX.J be fractional ideals; set 



G = {Ee 



1 



O 7 



J- 



! : det£ = 1 }. 



symmetric coprime lower pair for T(0, ... , C,X; J"). 



are a 
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(a) Let Q be a prime ideal, and choose A G X, a G Jd 1 such that 
ordQX = ordgl, ord^a = ordQjd~ x . Then there is some E G Q so that 

CE = (Co, 0) (mod Q) where Cq is nxr with rankQCo = r. 

(b) There is a matrix M G F(0, . . . ,0,1;J) so that (C\D)M = (C'\D') with 
detC",detir ^ and (( Jd' 1 ) 11 ! 2 det C , det D') = 1. 

/A £? 

(c,) There are n x n matrices A,B so that 



C D 



Proof. It suffices to establish the claims for F(0, ... ,0,V;^T) where V is a prime 
ideal in clsX. Choose a G Jd~ x so that ordgct = ordgj7"<9 _1 whenever ordg,7<9 -1 7^ 
0. Choose XeV -V 2 , /jE V' 1 - O so that A// = 1 (mod V). 



(a) Let C = a 



A 



C 



A 



(ci • • • c n ) (so is the ith column of C). 



Then there is some E' G GL n -i(0) so that 

&...£„_!)£;= (C£,0) (mod 7?) 

where Cg is n x r', r' = rank-p(c 1 . . . c n _ 1 ). So Ei 



G and 



CK 



A 



CEi 



I 



A 



^ = (^,0,^) (mod V). 



First suppose c n G span-pCg. Thus there exist 71, . . . , 7 r G O so that Cg7 



— c n (mod P) where 7 



CM 



A 



CE 1 



A 



/7i\ 

\lr) 

= CE 1 
= CE 1 



Let E = E\ 



(M) 



-1 



(C' ,0) (mod 7>), 




(mod P) 



proving (a) in the case c n G span-pCg. 

So suppose c n G" span^CQ. If r' = n — 1 then we are done, as then rank-pC = n 
and we can take E = I. So let us also suppose r' < n — 1. Choose 5 G V so 
that for all primes <2|/(/A, <2 /f<5. Thus (/iA, <5) = 1, so there are it, i> G O so that 

7 



t><5 — ufi\ = 1. Thus E 2 
I 



v m I G Q, and 
A 5 



a 



A 



CE1E2 



A 



Eo 



A 



(mod V) 
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where = c t for i < n — 1, d n -\ = c n (mod V) and d n = (mod V). 

Let i?3 = ( * ^ \ be a, permutation matrix that interchanges columns r' and 
n — 1. So -E = E1E2E3 G and (a) is proved. 

(b) Let A = (Jd' 1 ) 71 ■ J 2 • det C C 0. Take A e J, 77 e J" 1 , a e Jd" 1 , 7/ e J""^ 
so that Xt] = av = 1 (mod Q). 

Case 1: Say det D 7^ 0, detD ^ (9 X . Part (a) says there exists some E e Q so 
that 

a^ 1 A )^( J A ) =(Co,0) (mod Q), 



with Cq n x r, rankgCo = r. Thus 



! er(C,... ,0,1;J), and 



A 



(C|£>) 



/a/ 



V 



ctA 



\ 



(C , 0)£> , Di) (mod Q); 



77 



here .Do ^ spangCo (since C is symmetric; see the proof of Lemma 7.2 in [5]), 
and thus rankg(Col-Di) = n. 



Set W = [i 



M Q = 



0, 



E 



V 

I 



*e- 1 j yw 1 

Also, with (C'\D') = (C\D)M Q , detD" = detD and 



; hence 

er(o,... ,0,i-,j). 



fed 



aX 



\ 



X 



(C\D) 



E 



t E -i 



/al 



aX 



\ I' 



\ 



r I 

V 1 1/ 



(mod Q). 



Thus rank Q a ^ 1 x) °' ^ A ) = n ' and hence S /T( Jd' 1 )^ 2 det C 

Only finitely many prime ideals Q divide det D, so repeating this process for all 

such Q yields a pair (C'\D') = (C\D)M , M e T(0, . . . , C, J; J") , det C", det D' 7^ 0, 

and ((Jd" 1 ) 7 ^ 2 det C, detD') = 1. 

Case 2: Say det D e O x . If det C^O then we are done; so suppose det C = 0. 

Then let <2 be any prime ideal; following the algorithm in Case 1, we produce 
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M Q g T(0, . . . , 0,1; J) so that with (C'\D') = (C\D)M Q , we have det D' = det D 
and Q /detC" (so detC" 7^ 0). This completes this case. 

Case 3: Say det D = 0. Choose a prime Q; with A, 77, a, fx as in Case 1, we know 
there is some E G Q so that 



A 



(C|£>) 



/a/ 



V 



aX 



\ 



(Co,0|£> o ,£>i) (mod Q) 



7/ 



with rankg(Col-Di) = n, D Q C spangCo- Thus for some Y - G (9 r ' r , rankg(CoY + 
D G ,Di) = n. Note that Y is uniquely determined modulo Q, and that C *A) is 
symmetric modulo Q. Since C Y *C = —-Do *Co (mod Q), we can choose Y to be 
symmetric in O r,r . 
Set 

x)( Y o)( J Aj : 



hence M = 



E 



Y' = a 

I Y' 



t E -i 



G T(0, ... ,0,1;J). Thus with (C'\D') = 



(C\D)M, we have 



A 



{C'\D') 



/ al 



\ 



Oi\ 



\ 



= (C ,0|C y + ^o,^i) (mod Q). 



Hence rankg \j D ' y ^ J = n ' so ^ /detlT (and thus det-D' ^ 0). This 
reduces this situation to one of the previous cases. 

(c) Set k = det-D; choose A G A so that (k, A) = 1, and 77 G (9 
so that T] = k~ x (mod A) (this is possible by the Chinese Remainder The- 



orem). Since C G J x d 



Qr 



T 



-1 ' 



we have A *C 1 G 



. Also, A|(t7k — 1), so 



B = (r ]K -l) t C- 1 ejd' 1 x ^O n ' n (^ I 
(Note that locally everywhere: C G J^d (J j-i^j C o j-i ) , c o e C n ' n , 



det C = (Jd-^^detC; hence C" 1 G j^^d" 1 



J 



J 



•) 



Set A = We have k = det D, r) E O, so ??k *L> _1 G 



Qr 



--1 



Thus ( ^ ^ ) is a candidate for T(C, . . . ,0,1;J). 
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To see this matrix indeed lies in this group, first note that A t D — B l C = I, 
and by assumption, C l D is symmetric. Finally, substituting for A and B, we get 
A t B — r]K(r]K — l) t D~ 1 C~ 1 ; since t D~ 1 C~ 1 = (C t D)~ 1 is symmetric, so is A l B. 
□ 
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